fix: Dataproc operators fail to import without OpenLineage #46561
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
|
Nice. Some mypy checks but it looks good :) |
potiuk
approved these changes
Feb 7, 2025
Member
|
Quick fix @kacpermuda :) |
Signed-off-by: Kacper Muda <mudakacper@gmail.com>
kacpermuda
force-pushed
the
fix-google-ol-import-error
branch
from
8d9f817 to
682ff6c
Compare
Contributor
Author
|
@potiuk let's hold on with merging until we get @mobuchowski approval. We discussed few ways to solve this, and I want to hear his opinion. |
Member
|
Sure |
mobuchowski
approved these changes
Feb 7, 2025
Contributor
|
No need to be a single point of failure 🙂 |
sean-rose
added a commit
to mozilla/telemetry-airflow
that referenced
this pull request
Mar 26, 2025
…g present in v12.0.0. The bug was Dataproc operators failing to import without OpenLineage, fixed in apache/airflow#46561.
sean-rose
added a commit
to mozilla/telemetry-airflow
that referenced
this pull request
Mar 31, 2025
* feat: Upgrade Airflow from 2.9.3 to 2.10.5. * fix: Replace use of removed `SimpleHttpOperator` with `HttpOperator`. * fix: Upgrade `apache-airflow-providers-google` to v14.0.0 to fix a bug present in v12.0.0. The bug was Dataproc operators failing to import without OpenLineage, fixed in apache/airflow#46561. * fix: Replace uses of removed Dataproc job operators with `DataprocSubmitJobOperator`. * fix: ruff format modified Python files. * chore: Opt out of Airflow's usage telemetry in the local Docker Compose setup.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
related: #46478
This PR does three things:
BigQueryJobRunFacetto a separate file and add try/except on class definition, in case OpenLineage is not available (similar logic as in common.compat provider). I've also moved the import of this facet into specific method in _OpenLineageMixin, so it's no longer top level import.I've tested it manually on the environment without OpenLineage and the error described in the issue is gone after applying changes included in this PR.
CC @mobuchowski
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.